1. Field of the Invention
The present invention relates to an image processing of a picked-up image, and, in particular, relates to a technology that corrects a deterioration of the picked-up image due to an image pickup optical system with high accuracy.
2. Description of the Related Art
An object captured by an image pickup optical system has a minute extension because light generated from one point cannot be concentrated and has a minute extension, due to the influence of diffraction, aberration or the like, which are caused in the image pickup optical system. Such a distribution with minute extension is referred to as “PSF” (Point Spread Function). The picked-up image is formed as PSF is convoluted with an image because of the influence of the image pickup optical system, and the image blurs and the resolution is deteriorated.
Recently, it has become more common to hold the picked-up image as electronic data, and a technology to correct image deterioration due to an optical system by an image processing is well-known. Hereinafter, such image deterioration correction technology is referred to as “image restoration”, and a traditional and general method will be described.
When, on a real space (x, y), an image before the deterioration due to the optical system is represented as f(x, y), PSF is represented as h(x, y), and a deteriorated image is represented as g(x, y), these can be described by the following expressions (1).g(x, y)=∫∫f(X, Y)*h(x−X, y−Y)dXdY  (1)
When the above expression (1) is converted from the real space (x, y) into a frequency space (u, v) by performing a Fourier transformation to the above expression (1), it can be described as the following expression (2).G(u, v)=F(u, v)*H(u, v)  (2)
F(u, v) denotes a Fourier transform of f(x, y), G(u, v) denotes a Fourier transform of g(x, y), and H(u, v) denotes a Fourier transform of h(x, y).
Further, when the above expression (2) is transformed, the following expression (3) is derived.F(u, v)=G(u, v)/H(u, v)  (3)
This denotes that F(u, V), which is a Fourier transform of f(x, y) being the image before the deterioration, can be obtained by dividing G(u, v), which is a Fourier transform of g(x, y) being the deteriorated image, by H(u, v), which is a Fourier transform of h(x, y) being the PSF. Accordingly, if an inverse Fourier transformation is performed on F(u, v), f(x, y) being the image before the deterioration can be obtained.
However, when such a processing is actually performed to obtain the image before the deterioration, a noise that may be caused by an image pickup element is greatly amplified and a good image cannot be obtained. In this case, it is known to use a Wiener filter which is represented as the following expression (4), as an image restoration method for suppressing the amplification of the noise. H(u, v) denotes OTF (Optical Transfer Function), and Γ denotes a constant for reducing the amplified amount of the noise.W(u, v)=1/H(u, v)*|H(u, v)|2/(|H(u, v)|2+Γ)  (4)
When the above expression (4) is multiplied to the Fourier transform G(u, v) of the deteriorated image g(x, y), a good image can be obtained with high resolution by setting to 0 a phase component of a PSF generated by the diffraction or the aberration of the optical system and by amplifying a frequency characteristic of an amplitude component. Therefore, when frequency space information of an image restored by an image restoration processing using the Wiener filter is defined as R(u, v), the following expression is obtained.R(u, v)=G(u, v)*W(u, v)  (5)
As a result, a relationship of |W(u, v)|≦1/|H(u, v)| can be derived from the expression (4). In other words, the maximum value of the frequency characteristic that is amplifiable using the image restoration filter is 1/|H(u, v)|, and it equals to an inverse of an absolute value of OTF, that is to say, a characteristic of an inverse filter.
For example, when, for an image picked-up by a general camera or the like, the image restoration processing is performed by a processing using the Wiener filter of the expression (4), a correction value of the restoration processing can be controlled from 0 to 1/|H(u, v)| by controlling the value of Γ in the expression. Moreover, even if the Wiener filter is variously changed, the correction value of the restoration processing can be controlled with a range from 0 to 1/|H(u, v)|. In this embodiment, the range from 0 to 1/|H(u, v)| is defined as a correctable range (hereinafter also referred to as “settable range”).
When a computer where an image processing apparatus, an image pickup apparatus, or software is installed performs the processing using a correction value set by an instruction of a user, it is preferred to select or change a correction value according to a desired restoration level.
For example, Japanese Patent No. 3840032 discloses an invention that changes a range of a correction level of a correction value, such an edge emphasis, in a field of an image processing or the like. Moreover, Japanese Patent Laid-Open No. 2011-22868 discloses an invention that detects a restoration limit based on an image and a frequency characteristic of an image output device and performs an image restoration processing.
However, the prior arts disclosed in Japanese Patent No. 3840032 and Japanese Patent Laid-Open No. 2011-22868 do not describe a method for determining a concrete range of a level of the correction value in a system capable of combining various conditions. An image restoration processing used in digital cameras is greatly different from a general edge emphasis processing or a sharpness processing, and changes a correctable range with each combination of image pickup conditions, such an identification information of an image pickup optical system, a focal length, an F-number, an image pickup distance, an image height, and an pixel size of an image pickup element. Even if the image restoration processing is performed using a correction value beyond the correctable range, the user cannot experience the restoration effect.
Further, in another problem, when the correctable range is displayed on a display part, the maximum value of the correctable range when the restoration effect can be obtained is not displayed so that the user can recognize it, and therefore it is inconvenient when the correction value of the restoration processing is determined.